Skip to content

feat: add fireworks_ai to native OpenAI-compatible provider routing - #8803

Open
Felix Shi (shi-felix) wants to merge 3 commits into
crewAIInc:mainfrom
shi-felix:feat/fireworks-native-provider
Open

Felix Shi (shi-felix) wants to merge 3 commits into
crewAIInc:mainfrom
shi-felix:feat/fireworks-native-provider

Conversation

@shi-felix

@shi-felix Felix Shi (shi-felix) commented Oct 30, 2026 •

Copy link
Copy Markdown

Summary

fireworks_ai now routes through the native OpenAICompatibleCompletion instead of falling back to LiteLLM, so LLM(model="fireworks_ai/accounts/fireworks/models/<model-id>") works without the LiteLLM extra.

Fixes #8010.

Changes

  • lib/crewai/src/crewai/llms/providers/openai_compatible/completion.py: add the fireworks_ai ProviderConfig (https://api.fireworks.ai/inference/v1, FIREWORKS_API_KEY, optional FIREWORKS_BASE_URL) and list it in the class docstring.
  • lib/crewai/src/crewai/llm.py: add fireworks_ai to SUPPORTED_NATIVE_PROVIDERS, PROVIDER_ALIASES, the native-provider set in _get_native_provider, and _matches_provider_pattern.
  • lib/crewai/tests/llms/openai_compatible/test_openai_compatible.py: coverage for the registry entry, the missing-key error (and that an OpenAI key is not reused), prefixed and explicit-provider routing, FIREWORKS_BASE_URL, explicit-config precedence, account-scoped model ids, and the resolved client params.
  • docs/edge/{en,ar,ko,pt-BR}/concepts/llms.mdx: a Fireworks AI accordion.

Only the fireworks_ai/ prefix is stripped: an id such as
fireworks_ai/accounts/acme/models/qwen3-235b-a22b-instruct-2507 reaches the SDK as
accounts/acme/models/qwen3-235b-a22b-instruct-2507. The existing fireworks_ai/ spelling is kept, and no fireworks/ alias is added.

is_litellm=True still forces the LiteLLM path, and the other native providers are untouched.

Testing

python -m pytest lib/crewai/tests/llms/openai_compatible/test_openai_compatible.py -p no:randomly -q -n0 --basetemp=...
73 passed in 24.63s
python -m ruff check <changed files>          # All checks passed!
python -m ruff format --check <changed files> # 2 files already formatted

Python 3.13.14 on Windows. No provider credentials and no paid API calls were used.

AI assistance

Written with AI assistance (WorkBuddy) and reviewed before submission.


Note

Low Risk
Scoped LLM routing and provider config with tests; no auth or data-path changes beyond using the correct Fireworks API key env var.

Overview
Fireworks AI now uses CrewAI’s native OpenAICompatibleCompletion path instead of LiteLLM, so LLM(model="fireworks_ai/...") works without crewai[litellm].

Routing registers fireworks_ai in native provider lists and aliases, maps it to the shared OpenAI-compatible client with FIREWORKS_API_KEY, optional FIREWORKS_BASE_URL (default https://api.fireworks.ai/inference/v1), and requires Fireworks credentials—OPENAI_API_KEY is not reused. Only the fireworks_ai/ prefix is stripped; paths like accounts/<account>/models/<model> are sent to the API unchanged.

Tests cover config, routing, env overrides, explicit api_key/base_url, and client params. Docs in en/ar/ko/pt-BR add a native Fireworks accordion (mirroring DeepInfra); older LiteLLM-only Fireworks sections elsewhere in those pages are not updated in this diff.

Reviewed by Cursor Bugbot for commit a60d5ed. Bugbot is set up for automated code reviews on this repo. Configure here.

@mintlify

mintlify Bot commented Oct 30, 2026 •

Copy link
Copy Markdown

Preview approval required

This pull request comes from a fork, so Mintlify won't build a preview until someone with write access approves commit e7fe0d7.

Approve preview

@strix-security

strix-security Bot commented Oct 30, 2026 •

Copy link
Copy Markdown

Strix Security Review

Warning

This pull request has 117 commits after the last Strix review (f5d5ffa). Strix has not reviewed these changes.
Automatic review on push is off for this repository. To review the latest changes, tag @strix-security in a comment, or turn on re-review on push.

✅ No open security findings.

PR summary: Adds `fireworks_ai` as a native OpenAI-compatible provider so `LLM(model="fireworks_ai/accounts/<acct>/models/<model>")` routes through the built-in OpenAI client instead of LiteLLM, with a provider config, routing registration, tests, and translated docs.

Review summary

Reviewed all 7 changed files: the new fireworks_ai ProviderConfig, the four routing registrations in crewai/llm.py, the added tests, and the four translated docs accordions. The change is additive and narrow — it only appends a provider entry and routes the fireworks_ai/ prefix to the existing native OpenAI-compatible client. Credential handling was the main concern, and the new entry is stricter than the LiteLLM path it replaces: it requires FIREWORKS_API_KEY explicitly (no OPENAI_API_KEY fallback, since the subclass validator resolves or raises before the parent's fallback runs), and the base URL resolves to an explicit argument, FIREWORKS_BASE_URL, or the fixed Fireworks endpoint, never to the OpenAI envelope defaults. Model-id handling is safe: PROVIDER_ALIASES is an exact dict lookup, the model is split on the first slash only so accounts/<acct>/models/<model> ids survive intact, and the model string reaches the SDK solely as a JSON body field, giving no control over host, path, or headers. Two base-URL concerns were raised and independently rejected as pre-existing, operator-controlled configuration on unchanged code. No security issues found in the changed code.

Ruled out (4)
  • Can a crafted model id be misrouted by the new fireworks_ai registration or inject into the request URL, host, or headers?
    Provider detection uses an exact `PROVIDER_ALIASES` dict lookup and `partition("/")` on the first slash only, and the model string reaches the SDK solely as the JSON body field, so nested ids cannot influence the request target.
  • Can a missing FIREWORKS_API_KEY fall back to OPENAI_API_KEY and send an OpenAI credential to Fireworks?
    `_resolve_api_key` requires `FIREWORKS_API_KEY` and the subclass `mode="before"` validator runs ahead of the parent's `OPENAI_API_KEY` fallback, so the constructor raises `MissingAPIKeyError` first.
  • Can FIREWORKS_BASE_URL or an explicit base_url redirect the Fireworks API key to an arbitrary host?
    The value comes only from an explicit constructor kwarg or the operator-set env var, the same documented proxy knob the eight pre-existing provider entries already expose, so no trust boundary is crossed.
  • Can a stale or foreign OPENAI_BASE_URL in the environment redirect a fireworks_ai request and leak the Fireworks key?
    That provider-agnostic env read lives in unchanged `utilities/llm_utils.py` and ships identically on main, and anyone able to set it can already read the key from the same environment.

Review 1 on this PR · f5d5ffa · requested Oct 30 05:43 UTC when the PR was opened


Reviewed by Strix
Re-run review · Configure security review settings

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Want higher recall? High effort reviews run extra passes and find more bugs. A team admin can switch effort levels in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit f5d5ffa. Configure here.

api_key_env="FIREWORKS_API_KEY",
base_url_env="FIREWORKS_BASE_URL",
api_key_required=True,
),

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fireworks context windows miss catalog

Medium Severity

Native fireworks_ai routing inherits OpenAICompletion.get_context_window_size, which queries OPENAI_CONTEXT_WINDOWS only. Catalogued Fireworks limits never match, so those models compact against the 8192 default instead of their published windows.

Additional Locations (1)
Fix in Cursor Fix in Web

Triggered by learned rule: Context-window catalog matching

Reviewed by Cursor Bugbot for commit f5d5ffa. Configure here.

@shi-felix

Copy link
Copy Markdown
Author

Cursor Bugbot is right that Fireworks ids never match OPENAI_CONTEXT_WINDOWS — and the same holds for every other compatible provider, not just this one: DeepInfra has 272 catalogued ids and Groq has 18, all equally unreachable from a table filtered to litellm_provider == "openai".

I've opened #8812 to resolve compatible-provider context windows from their own catalog entries. Once it lands, this PR (and #8801/#8804) pick the fix up automatically — groq, fireworks_ai and together are already in its provider set. Gryphe/MythoMax-L2-13b on DeepInfra is a concrete before/after there: catalogued at 4,096, previously resolved to the 8,192 default.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[FEATURE] Add Fireworks AI to native OpenAI-compatible provider routing

1 participant